Deep Learning and Deep Reinforcement Learning for Graph Based Applications

Dyp læring har gitt state-of-the-art ytelse i mange applikasjoner som datasyn, tekstanalyse, biologi, osv. Suksessen med dyp læring har også hjulpet fremveksten av dyp forsterkende læring for optimal beslutningstaking og har vist stort potensiale, spesielt i optimaliseringsproblemer. I tillegg har grafer som matematisk representasjon for strukturerte komplekse systemer vist seg å være et kraftig verktøy for analyse og problemløsning, og gitt et nytt perspektiv på formuleringen av problemet. Ved å introdusere grafer som en inputmodalitet for maskinlæringsproblemer kan dyplæringsmodeller enten bruke strukturen til grafen i sine representasjonslæringsskjema, eller optimalisere grafstrukturen i en nedstrøms evalueringsoppgave. Dette vil også føre til modellmetoder og pipelines som utnytter den strukturelle informasjonen gitt av grafer til forbedret ytelse, sammenlignet med tradisjonelle maskinlæringsmodellers kapasitet. I denne oppgaven introduserer vi fem forskjellige use-case-applikasjoner, gjennom fem forskningsartikler, som kan modelleres som grafer og tar sikte på å skape nye modeller som adresserer problemer ved bruk av dyp grafrepresentasjonslæring og dype forsterkningslæringsmodeller. Våre tre viktigste applikasjonsdomener er bioinformatikk, datasyn og logistikk. Først tar vi sikte på å adressere to problemer innen bioinformatikk. I Paper I tar vi opp spørsmålet om integrering av kontinuerlige omics-datasett med biologiske nettverk. Vi introduserer et auto-koderskjema fokusert på representasjonslæring av nodefunksjoner i biologiske nettverk, og viser anvendelsen av det utformede rammeverket i et virkelighetseksempel gjennom imputering av manglende verdier i et eksempeldatasett for omics. Paper II ser på bruk av grafrepresentasjonslæring for å behandle metabolske nettverk. I den foreslåtte tilnærmingen introduserer vi en maskinlæringspipeline (fra funksjonsekstraksjon til modellarkitektur) basert på grafiske nevrale nettverk og evaluerer pipelinen basert på prediksjon av genessensalitet, som er en velkjent bruk av metabolske banenettverk. Det andre domenet av applikasjoner er datasynsdomenet, spesifikt problemet med gjenkjennelse av menneskelige gester. I Paper III, og oppfølgingen Paper IV, introduserer vi et gestgjenkjenningssystem som er både raskere og mer nøyaktig enn den avanserte prediksjonen av menneskelige motivbevegelser fra mmWave Radar genererte punktskyer. Vi oppnår dette ved å modellere inngangspunktskyen som en spatio-temporal graf og å bearbeide den opprettede grafen ved bruk av den foreslåtte læringsteknikken for grafrepresentasjon. Videre evaluerer vi systemet under forskjellige eksperimentelle forhold ut ifra vinkelen til emnet med hensyn til sansing, og foreslår en ensembletilnærming for å dempe effekten av å endre sansevinkelen på ytelsen til modellen. Den siste applikasjonen vi tar for oss er bruken av dyp forsterkningslæring for å optimalisere strukturen til grafene i kombinatoriske optimaliseringsproblemer i logistikk. Paper V introduserer en generell problemuavhengig hyperheuristikk som utnytter beslutningsevnen til dyp forsterkende læring, ved å bruke en problemuavhengig tilstandsfunksjonsinformasjon. Det foreslåtte rammeverket er trent på en generell belønningsfunksjon for å oppnå høykvalitets ytelse blant populære løsere innen kombinatorisk optimalisering. Vi evaluerer ytelsen til den foreslåtte tilnærmingen med tre eksempler på ruting problemer samt et planleggingsproblem, for å vise effektiviteten til metoden vår i forskjellige typer problemstillinger.Deep learning has provided state-of-the-art performance in many applications such as computer vision, text analysis, biology, etc. The success of deep learning has also helped with the emergence of deep reinforcement learning for optimal decision-making and has shown great promise, especially in optimization problems. Additionally, graphs as a mathematical representation for structured complex systems have proven to be a powerful tool for analysis and problem-solving that offer a fresh perspective on the formulation of the problem. Introducing graphs as an input modality for machine learning problems enables deep learning models to either utilize the structure of the graph in their representation learning scheme or optimize the graph structure for a downstream evaluation task. Doing so will also lead to model methods and pipelines that leverage the structural information provided by graphs to improve performance compared to traditional machine learning models. In this thesis, we introduce five different use-case applications, in the format of five research papers, that can be modeled as graphs and aim to provide novel models that address problems using deep graph representation learning and deep reinforcement learning models. Our main three application domains are bioinformatics, computer vision, and logistics. First, we aim to address two problems in the domain of bioinformatics. In Paper I, we address the issue of integration of continuous omics datasets with biological networks. We introduce an auto-encoder scheme focused on representation learning of node features in biological networks and showcase the application of the designed framework in a real-world example through the imputation of missing values in an example omics dataset. Paper II looks at utilizing graph representation learning for processing metabolic networks. In the proposed approach, we introduce a machine learning pipeline (from feature extraction to model architecture) based on graph neural networks and evaluate the pipeline on the task of gene essentiality prediction which is a well-known application of metabolic pathway networks. The second domain of applications is the computer vision domain specifically the problem of human gesture recognition. In Paper III and the follow-up Paper IV, we introduce a gesture recognition system that is both faster and more accurate compared to the state-of-the-art prediction of human subject gestures from mmWave Radar generated point clouds. We achieve this by modeling the input point cloud as a spatio-temporal graph and processing the created graph using the proposed graph representation learning technique. We further evaluate the system in different experimental conditions in terms of the angle of the subject with respect to sensing and propose an ensemble approach for mitigating the effect of changing the sensing angle on the performance of the model. The last application that we address is the use of deep reinforcement learning to optimize the structure of the graphs in combinatorial optimization problems in logistics. Paper V introduces a general problem-independent hyperheuristic that utilizes the decision-making capability of deep reinforcement learning using a problem-independent state feature information. The proposed framework is trained on a general reward function to achieve state-of-the-art performance among popular solvers in the field of combinatorial optimization. We evaluate the performance of the proposed approach on three example routing problems as well as a scheduling problem to showcase the effectiveness of our method in different problems.Doktorgradsavhandlin

A Graph Feature Auto-Encoder for the Prediction of Unobserved Node Features on Biological Networks

Background Molecular interaction networks summarize complex biological processes as graphs, whose structure is informative of biological function at multiple scales. Simultaneously, omics technologies measure the variation or activity of genes, proteins, or metabolites across individuals or experimental conditions. Integrating the complementary viewpoints of biological networks and omics data is an important task in bioinformatics, but existing methods treat networks as discrete structures, which are intrinsically difficult to integrate with continuous node features or activity measures. Graph neural networks map graph nodes into a low-dimensional vector space representation, and can be trained to preserve both the local graph structure and the similarity between node features. Results We studied the representation of transcriptional, protein–protein and genetic interaction networks in E. coli and mouse using graph neural networks. We found that such representations explain a large proportion of variation in gene expression data, and that using gene expression data as node features improves the reconstruction of the graph from the embedding. We further proposed a new end-to-end Graph Feature Auto-Encoder framework for the prediction of node features utilizing the structure of the gene networks, which is trained on the feature prediction task, and showed that it performs better at predicting unobserved node features than regular MultiLayer Perceptrons. When applied to the problem of imputing missing data in single-cell RNAseq data, the Graph Feature Auto-Encoder utilizing our new graph convolution layer called FeatGraphConv outperformed a state-of-the-art imputation method that does not use protein interaction information, showing the benefit of integrating biological networks and omics data with our proposed approach. Conclusion Our proposed Graph Feature Auto-Encoder framework is a powerful approach for integrating and exploiting the close relation between molecular interaction networks and functional genomics data.publishedVersio

A general deep reinforcement learning hyperheuristic framework for solving combinatorial optimization problems

Many problem-specific heuristic frameworks have been developed to solve combinatorial optimization problems, but these frameworks do not generalize well to other problem domains. Metaheuristic frameworks aim to be more generalizable compared to traditional heuristics, however their performances suffer from poor selection of low-level heuristics (operators) during the search process. An example of heuristic selection in a metaheuristic framework is the adaptive layer of the popular framework of Adaptive Large Neighborhood Search (ALNS). Here, we propose a selection hyperheuristic framework that uses Deep Reinforcement Learning (Deep RL) as an alternative to the adaptive layer of ALNS. Unlike the adaptive layer which only considers heuristics’ past performance for future selection, a Deep RL agent is able to take into account additional information from the search process, e.g., the difference in objective value between iterations, to make better decisions. This is due to the representation power of Deep Learning methods and the decision making capability of the Deep RL agent which can learn to adapt to different problems and instance characteristics. In this paper, by integrating the Deep RL agent into the ALNS framework, we introduce Deep Reinforcement Learning Hyperheuristic (DRLH), a general framework for solving a wide variety of combinatorial optimization problems and show that our framework is better at selecting low-level heuristics at each step of the search process compared to ALNS and a Uniform Random Selection (URS). Our experiments also show that while ALNS can not properly handle a large pool of heuristics, DRLH is not negatively affected by increasing the number of heuristics.publishedVersio

A Graph Feature Auto-Encoder for the Prediction of Unobserved Node Features on Biological Networks

Background Molecular interaction networks summarize complex biological processes as graphs, whose structure is informative of biological function at multiple scales. Simultaneously, omics technologies measure the variation or activity of genes, proteins, or metabolites across individuals or experimental conditions. Integrating the complementary viewpoints of biological networks and omics data is an important task in bioinformatics, but existing methods treat networks as discrete structures, which are intrinsically difficult to integrate with continuous node features or activity measures. Graph neural networks map graph nodes into a low-dimensional vector space representation, and can be trained to preserve both the local graph structure and the similarity between node features. Results We studied the representation of transcriptional, protein–protein and genetic interaction networks in E. coli and mouse using graph neural networks. We found that such representations explain a large proportion of variation in gene expression data, and that using gene expression data as node features improves the reconstruction of the graph from the embedding. We further proposed a new end-to-end Graph Feature Auto-Encoder framework for the prediction of node features utilizing the structure of the gene networks, which is trained on the feature prediction task, and showed that it performs better at predicting unobserved node features than regular MultiLayer Perceptrons. When applied to the problem of imputing missing data in single-cell RNAseq data, the Graph Feature Auto-Encoder utilizing our new graph convolution layer called FeatGraphConv outperformed a state-of-the-art imputation method that does not use protein interaction information, showing the benefit of integrating biological networks and omics data with our proposed approach. Conclusion Our proposed Graph Feature Auto-Encoder framework is a powerful approach for integrating and exploiting the close relation between molecular interaction networks and functional genomics data

Integration of graph neural networks and genome-scale metabolic models for predicting gene essentiality

Abstract Genome-scale metabolic models are powerful tools for understanding cellular physiology. Flux balance analysis (FBA), in particular, is an optimization-based approach widely employed for predicting metabolic phenotypes. In model microbes such as Escherichia coli, FBA has been successful at predicting essential genes, i.e. those genes that impair survival when deleted. A central assumption in this approach is that both wild type and deletion strains optimize the same fitness objective. Although the optimality assumption may hold for the wild type metabolic network, deletion strains are not subject to the same evolutionary pressures and knock-out mutants may steer their metabolism to meet other objectives for survival. Here, we present FlowGAT, a hybrid FBA-machine learning strategy for predicting essentiality directly from wild type metabolic phenotypes. The approach is based on graph-structured representation of metabolic fluxes predicted by FBA, where nodes correspond to enzymatic reactions and edges quantify the propagation of metabolite mass flow between a reaction and its neighbours. We integrate this information into a graph neural network that can be trained on knock-out fitness assay data. Comparisons across different model architectures reveal that FlowGAT predictions for E. coli are close to those of FBA for several growth conditions. This suggests that essentiality of enzymatic genes can be predicted by exploiting the inherent network structure of metabolism. Our approach demonstrates the benefits of combining the mechanistic insights afforded by genome-scale models with the ability of deep learning to infer patterns from complex datasets

Tesla-Rapture

Publisher Copyright: IEEE | openaire: EC/H2020/813999/EU//WINDMILLWe present Tesla-Rapture, a gesture recognition system for sparse point clouds generated by mmWave Radars. State of the art gesture recognition models are either too resource consuming or not sufficiently accurate for the integration into real-life scenarios using wearable or constrained equipment such as IoT devices (e.g. Raspberry PI), XR hardware (e.g. HoloLens), or smart-phones. To tackle this issue, we have developed Tesla, a Message Passing Neural Network (MPNN) graph convolution approach for mmWave radar point clouds. The model outperforms the state of the art on three datasets in terms of accuracy while reducing the computational complexity and, hence, the execution time. In particular, the approach, is able to predict a gesture almost 8 times faster than the most accurate competitor. Our performance evaluation in different scenarios (environments, angles, distances) shows that Tesla generalizes well and improves the accuracy up to 20% in challenging scenarios, such as a through-wall setting and sensing at extreme angles. Utilizing Tesla, we develop Tesla-Rapture, a real-time implementation using a mmWave Radar on a Raspberry PI 4 and evaluate its accuracy and time-complexity. We also publish the source code, the trained models, and the implementation of the model for embedded devices.Peer reviewe